Learning Optical Flow from Real Robot Data

This project presents a method for teaching robot arms optical flow. By extending recent works on depth based, probabilistic robot tracking, a novel real-world optical flow dataset with dense ground truth annotation is generated. A real, robotic dataset boasts the inclusion of phenomena that synthetic datasets cannot model. With this data a generic convolutional neural network is implemented to predict optical flow. Current performance, with limited computational resources available, struggles to match the performance of classical techniques (Lucas & Kanade, Horn & Schunck, etc). Positive long-term results however would have strong implications in the field of robotic manipulation and grasping – allowing robots to generate and learn optical flow autonomously while also developing a methodology for understanding dynamic scenes. Keywords-optical flow, machine learning, robotics, deep learning, computer vision, convolutional neural nets